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ABSTRACT 



The Naval Postgraduate School is conducting a three-year study to determine 
whether or not to use the Graduate Record Examination as a selection tool, admissions 
standard. Students are currently selected based on their professional militar\’ perform- 
ance and their undergraduate grades, math and science exposure. This thesis examines 
a sample of 198 students who took the Graduate Record Examination after arrival at the 
school and who have completed six quarters of study. The results indicate that the 
Graduate Record Examination is a much stronger predictor than the currently used 
undergraduate measures (Academic Profile Code). When the Graduate Record Exam- 
ination scores are combined with undergraduate grade point average and the olTicer's 
age, an excellent predictor is developed. The thesis contains, in addition to descriptive 
information and regression results, a prediction equation which may be used by Na ^7 
selection committees in determining whether or not an officer will succeed at the Naval 
Postgraduate School. 
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I. INTRODUCTION 



[The Na\'\] requires ofTicers capable of original thought and the capacity to synthe- 
size broad areas of knowledge, analyze complex issues, and appreciate the dis- 
tinction between what is theoretically possible and actually achievable. Investment 
in graduate education must be pursued as a priority, even in the face of fiscal aus- 
terity and competing demands for our junior olTicers. 

Admiral Carlisle H. Trost 
[Ref 1: p. 4] 

To carr\' out this Chief of Naval Operations Policy, the Na%7 must select those of- 
ficers for graduate education who will succeed in their academic endeavor and benefit the 
service. Most olTicers selected attend the Naval Postgraduate School in Monterey, 
California. There they study a variety of technical, scientific and management curricula. 

A. ADMISSION STANDARDS 

The Naval Postgraduate School, as all other graduate schools, seeks to enroll those 
students who have the best chance for success in their academic endeavor and their re- 
lated professional careers. It is impossible to accurately predict the future, but measures 
are available for assessing academic potential. Among these are the Graduate Record 
Examination (GRE), the Graduate Management Admission Test (G.MAT), undergrad- 
uate grade point averages, personal recommendations and resumes of extracurricular 
and community activities. 

The Chief of Naval Operations' policy is that graduate education be afforded those 
officers "who have demonstrated superior professional performance and the intellectual 
capability to complete a rigorous academic program." [Ref 1: p. 4] The Na\ 7 ’, therefore, 
assesses a prospective student's academic potential through professional militar>' per- 
formance, potential for further promotion and the Academic Profile Code. The Aca- 
demic Profile Code is actually a combination of assessments: undergraduate grade point 
average and undergraduate exposure to math and science. 

The Navy is interested in the best match between individuals and education pro- 
grams it can achieve. This is particularly true when it comes to graduate education for 
its officers. A great deal is invested in the Naval Postgraduate School and the officers 
who attend. Beyond the obvious costs of maintaining an institution on a par with 
prestigious civilian graduate schools, there are opportunity costs to the Navw in having 
officers attending school instead of performing their militar}' missions in the fleet. 



1 



The question is: Do the Na\T's current admission criteria provide the best match 
between individual and curricular program? 

B. BACKGROUND 

The 19S4 Graduate Education Review Board met in October of that year to plan 
resolution of issues discussed at the meeting of the Graduate Education Review Group 
two days earlier. One issue of interest was graduate education selection and admission 
criteria. 

The Vice Chief of Naval Operations, in January’ 1985, requested "NTS OP-OI ex- 
plore the use of indices to measure the quality and potential of entering graduate stu- 
dents." [Ref 2] He further suggested that "national norms such as the Graduate Record 
Examinations should be considered [as] these factors could ultimately provide useful 
broad-based correlations on subsequent student academic performance and provide 
useful quality control data." [Ref 2] 

In response to this tasking, the Superintendent of the Naval Postgraduate School 
identified three deficiencies of the Academic Profile Code: 

1. Differences in institution standards cannot be measured. 

2. There is an engineering bias. 

3. Written and verbal skills are not quantified. [Ref 3] 

He further recommended the Graduate Record Examination be required of all offi- 
cers during college or during officer training. His recommendation discussed the benefits 
of the Graduate Record Examination: uniformity and data for correlation analysis re- 
garding admission criteria and success in graduate school. 

Another exchange of letters followed this initial round and culminated in June of 
1985 with approval of a three-year study of the Graduate Record Examination. This 
letter indicated that all officers ordered to the graduate program from April 1986 until 
April 1989 would be required to take the Graduate Record Examination before arrival 
at the school. The scores would be reported to Commander, Naval .Military Personnel 
Command for inclusion in the officer master file. This would allow Naval Postgraduate 
School to collect data for analysis. [Ref 4] 

A final correspondence exchange authorized administration of the Graduate Record 
Examination within the first two weeks of students' arrival at the Naval Postgraduate 
School. That is the form of the study which is currently undenvay. All U.S. Navy offi- 
cers take the Graduate Record Examination within the first few weeks of arrival at the 



school. Scores are subsequently reported to Commander, Naval Military Personnel 
Command and to the Naval Postgraduate School. 

C. RESEARCH QUESTIONS 

The questions addressed in this thesis are: 

1. Which criteria predict academic performance at the Naval Postgraduate School 
better: Academic Profile Code scores or Graduate Record Examination scores? 

2. Is there a difference in the predictive value for different curricula? 

3. How do the Naval Postgraduate School students compare with other graduate 
students at other institutions? 
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II. LITERATURE REVIEW 



Not much research has been done at the Naval Postgraduate School regarding al- 
ternate admission or selection criteria. Most efforts were class projects, staff specu- 
lations or studies directed toward a specific curriculum. There was apparently an effort 
to institute the Graduate Record Examination as a selection variable in the 1960s. A 
series of three theses was written during that time discussing the Graduate Record Ex- 
amination as well as other predictive instruments. All three, though focusing on the 
management curriculum, bear mention. Other studies conducted in the 1970s and 1980s 
are also of interest to this thesis. 

A. 1960'S 

1. Martz/Rushin 

In 1962, Martz and Rushin produced the first of this series of theses examining 
the use of the Graduate Record Examination in the management curriculum. They 
considered the Graduate Record Examination, the California .Analogies and Reasoning 
Test and the Na\y Officer Classification Battery. After performing a number of statis- 
tical tests on data collected from the 1962 class (N’ = 94), they determined the Graduate 
Record Examination was the most statistically significant of the instruments examined. 
Their reconunendation to use the Graduate Record Examination was couched in hesi- 
tant terms, however. Specifically, "the Graduate Record Examination (aptitude) 
produced by the Educational Testing Service was found to be the best of the three in- 
struments considered but is encumbered with certain restrictions that reduce the adapt- 
ability for Na^T-wide testing as proposed in this study. The Graduate Record 
E.xamination is recommended as a highly effective tool for faculty-student counselling 
and guidance programs." [Ref 5; p. i] 

2. Kauder/Ebert 

The second in the Management School theses series was written in 1963 by 
Kauder and Ebert. They studied the N'a\w Officer Classification Battery, the Graduate 
Record Examination and the Na\y Officer Qualification Test. Statistical analysis was 
conducted on the class of 1963 (N'=94) with a statistical reliability of 95 percent. 
Kauder and Ebert concluded that the Graduate Record Examination had a very high 
validity and was, in fact, the best predictor of the three options. Based upon this 
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conclusion, they recommended the Graduate Record Examination be used as the ad- 
mission criterion. [Ref. 6] 

3. Dreese/Russel 

Finally, in 1964, Dreese and Russel examined the Graduate Record Examina- 
tion, the Structured-Objective Rorschach Test-Sort, the Allport-Vernon-Lindzey Study 
of Values and a local questionnaire of motivation. They studied the management class 
of 1964 (N = 99). After extensive statistical correlation, they concluded that the Gradu- 
ate Record Examination was an "excellent predictor of academic performance" [Ref 7: 
p. 29] and should be administered to management candidates. [Ref 7] 

B. 1970'S 

1. Cook 

Worthy of mention is a thesis conducted in 1974 by Cook. This paper is con- 
cerned with the merits of the Graduate Record Examination, the Strong Vocational In- 
terest Blank, a biographical questionnaire and undergraduate academic performance in 
determining success in the Communications Management curriculum (N = 42). This 
thesis, unlike the previous studies, considered not only psychological and testing instru- 
ments, but also the Educational Potential Code. The Educational Potential Code was 
an earlier version of today's Academic Profile Code. His conclusion did not specify 
which of these instruments was best, but instead built a series of tables with combina- 
tions of indicators to assist in student selection. [Ref 8] 

2. Elster 

In August of 1974, Professor R. S. Elster prepared a letter to the Naval Per- 
sonnel Research and Development Center offering a manual to be used by the graduate 
education selection boards. It contained prediction tables for four curricula. These ta- 
bles were based on such factors as undergraduate grades, undergraduate university 
"school scores," age, foreign language ability, whether or not the officer was a Naval 
Academy graduate, rank. Graduate Record Examination scores and whether or not the 
officer had an engineering degree. The tables were to be used after the selection boards 
had considered professional military' performance and undergraduate academic 
performance. [Ref 9] 

C. 19S0'S 

1. Michealson/Phillips/Jeong/Lee 

In a 1985 class project, Michealson, Phillips, Jeong and Lee attempted specif- 
ically to look at correlations between final grades and the Academic Profile Code, as well 
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as final grades and undergraduate grade point averages. They studied the December 
1984 graduating class (N = 52). They found the highest correlations among those stu- 
dents who were Naval Academy graduates and those students who were in technical 
curricula. One would expect this result since the Acadenric Profile Code measures not 
only the undergraduate grade point average, but also exposure to technical curricula. 
[Ref. 10] 

2 . Blatt 

Blatt used an analysis of variance technique to look at students in the Oper- 
ations Analysis curriculum (N'= 159). He was interested in student performance as 
measured by the Academic Profile Code, time since undergraduate studies, which 
undergraduate college the student attended, what kind of degree the student earned, 
what the student's militarx' desginator was and whether or not the student had attended 
the math refresher training before starting the Operations Analysis courses. He found 
the undergraduate grade point average score of the Academic Profile Code to be a sig- 
nificant variable. The math and science codes of the Academic Profile Code, however, 
were not meaningful. Other factors he found to be significant were the time away from 
undergraduate studies, military designator and type of college degree. [Ref 11] 

3. Barr/ Howard 

Perhaps the most definitive and useful study done to date was accomplished by 
Barr and Howard in 1987. They took a preliminary look at data collected from the 
three-year study begun in April 1986. Their report encompassed 320 records of students 
who had taken the Graduate Record Examination and had completed at least three 
quarters of study at the Naval Postgraduate School. 

Their conclusions were five-fold: 

1. Using the Graduate Record Examination in conjunction with the other currently 
used admission criteria will significantly improve the prediction. 

2. The best selection of variables is the verbal and quantitative scores of the Graduate 
Record Examination used with the undergraduate grade point average score of the 
Acadeinic Profile Code and the student's age. 

3. The math and science scores of the .A.cademic Profile Code are not significantly 
useful in prediction. They are, however, still an important part of the admission 
criteria. 

4. Predictor significance varies over curricula. 

5. Distinguishing poor performers is difficult using the variables available. [Ref 12] 



6 



III. ASSUMPTIONS AND METHODOLOGY 



The priman' issue of this thesis is whether or not the Graduate Record Examination 
is a good predictor of success at the Naval Postgraduate School. What is success? How 
is it measured? This chapter will attempt to define success, the indices used to measure 
it and the methodology used to determine the applicability and strengths of those 
measures. 

A. "SUCCESS" AT THE NAVAL POSTGRADUATE SCHOOL 

How does one define success as related to academic performance? Grades, student 
involvement in campus affairs, the degree, quality of the thesis; all could be considered 
indicators of success. Hartnett and Willingham discuss how criteria for success are 
widely defined among schools, school departments, and even curricula within depart- 
ments. They offer three categories of criteria. 

1. Categories 

The first category is "administrative measures" such as grades, exams, the thesis 
and status of the degree. Second, "professional accomplishments" such as recognition 
through awards or other honors, publications, professional activities and experience. 
And last, specially developed "objective criteria" of the sort than can be measured such 
as work samples, common examinations or ratings of competencies and characteristics. 
[Ref 13: p. 10] 

The remainder of their study discusses merits and weaknesses of each of the 
criteria mentioned. Of particular interest to this study is their discussion of grades as a 
measure of success. 

2. Grades 

a. Advantages 

On a positive note, Hartnett and Willingham suggest that grades are an in- 
dication of the faculty's view of academic progress. They are readily available for all 
students in a scale easily interpreted by most. They also appear to be a composite in- 
dicator of variations in academic performance in a variety of courses. [Ref 13; pp. 11-12] 

b. Disadvantages 

There are a number of disadvantages in using grades as a predictor. Grades 
generally are of a verx' restricted range and do not reflect the potentially wide range of 
differences in student accomplishment. Standards are extremely variable over time. 
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institutions, departments and curricula. The basis of the grades is not always clear: 
exams, papers, research and class participation are all possibilities. [Ref. 13: pp. 12-13] 
They conclude "that while grades serve several useful functions in graduate 
education, the one served least well is that of providing an understandable criterion of 
graduate student performance." [Ref 13: p. 14] They acknowledge that grades are part 
of the evaluation process and should be used judiciously. 

3. Quality Point Rating as the Preferred Measure of Success 

Taking the above cautions under consideration, this study will use grade average 
as the criterion against which to measure success in graduate school. The Naval Post- 
graduate School term for grade average is "Quality Point Rating." 

The Quality Point Rating is a weighted average of grades computed using the 
grade values depicted in Table 1. 



Table 1. QUALITY POINT RATING COMPUTATION 



Grade 


Points 


A 


4.0 


A- 


3.7 


B + 


3.3 


B 


3.0 


B- 


2.7 


C + 


2.3 


C 


2.0 


C- 


1.7 


D + 


1.3 


D 


1.0 


X 

Source: Admissions Office Handout 


0.0 



The values in Table 1 are multiplied by the number of quarter-hour credits for 
each course. All course quality points are then sununed and subsequently divided by the 
total number of credit hours. The resulting number is the Quality Point Rating. The 
Quality Point Rating is calculated separately for total performance in all courses and for 
graduate-level courses. [Ref 14: pp. 20-21] 
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The Naval Postgraduate School defines success in graduate education as being 
eligible for the Master's Degree by attaining "a minimum average Quality Point Rating 
of 3.00 in all the 3000 and 4000 level [graduate) courses in (the) curriculum, and 2.5 in 
the remaining courses or a 2.75 in all courses of the curriculum." [Ref 14; p. 21) 

B. ADMISSION CRITERIA 
1. Academic Profile Code 

The Academic Profile Code is currently used in conjunction with professional 
militaiy’ performance as the most important admission criterion. It is composed of three 
digits reflecting the student's undergraduate experiences. 
a. Undergraduate Quality Point Rating Code 

This first digit of the Academic Profile Code describes the officer's Under- 
graduate Quality Point Rating. Table 2 displays the code, grades and Undergraduate 
Quality Point Rating range. 



Table 2. UNDERGRADUATE QUALITY POINT RATING CODE 



Code 


Grade 


UQPR Range 


0 


A- A 


3.60-4.00 


1 


B + 


3.20-3.59 


2 


B- B 


2.60-3.19 


3 


C + 


2.20-2.59 


4 


C 


1.90-2.19 


5 


Below C 


0.00-1.89 



Source: Admissions Office Handout 



Codes 0 through 3 must be based on a minimum of 100 semester or 150 
quarter hours of graded courses. Pass, fail, credit examinations and similar credits are 
not considered. 

b. Undergraduate Math Code 

The second digit of the Academic Profile Code indicates the officer's 
undergraduate exposure to math. A description of the codes follows in Table 3 on page 
10 . 
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Table 3. UNDERGRADUATE MATH CODE 



Code Calculus- Related Math Courses 

0 Significant post-calculus math with a B average 

1 Two or more calculus courses with a B+ average 

2 Two or more calculus courses with a C+ average 

3 One calculus course with a C grade or better 

4 Two or more pre-calculus courses with a B average or better 

5 At least one pre-calculus course with a C grade or better 

6 No college-level pre-calculus course with a C grade or better 

Source: Admissions Office Handout 



c. Undergraduate Technical Code 

The final digit of the Academic Profile Code reflects undergraduate experi- 
ence in physics or engineering. The codes are interpreted in Table 4. 



Table 4. UNDERGRADUATE TECHNICAL CODE 



Code 


Courses 


0 


Significant upper-division course coverage in a pertinent 
engineering or physical science discipline with a B -i- average 


1 


Significant upper-division course coverage in a pertinent 
engineering or physical science discipline with a C+ average 


2 


Complete calculus-based physics sequence with a B-i- average 


3 


Complete calculus-based physics sequence with a C-l- average 


4 


At least on calculus-based physics course with a C grade 


5 


None 


Source: Admissions Office Handout 
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2. Graduate Record Examination 

The Graduate Record Examination has been in use in one form or another since 
the late 1930s. This standardized paper and pencil test was first administered as a 
battery of eight exams measuring math, physics, chemistry, biology, social studies, liter- 
ature, fine arts and verbal skills. Advanced tests (known today as subject exams) were 
also developed to measure skill in very specific areas of study. [Ref 15: p. 2] 

The exams have been revised constantly over the years and the number of stu- 
dents taking the exam has increased as well. The format of the examinations adminis- 
tered during the Xav'al Postgraduate School study has been in use since 1981. It was 
during that year Educational Testing Service conducted a major revision of the General 
Test, particularly of the analytical measure. Until 1981, Educational Testing Service had 
cautioned against using the analytical measure for admission decisions. These cautions 
have since been removed. [Ref 16: p. 7] 

a. The Exam 

The General Test, which is being administered during the Naval Postgrad- 
uate School study, consists of three measures of "developed ability." [Ref 16: p. 6] 
Questions using antonyms, analogies, sentence completions and reading comprehension 
make up the verbal measure. Quantitative skills are examined through use of discrete 
quantitative questions, data interpretation and quantitative comparisons. Finally, the 
analytical section of the exam tests analytical reasoning and logical reasoning. [Ref 
16: p. 7] 

b. Scores 

Scores on these three measures are the total number of questions an 
examinee answered correctly in each section (the raw score) equated with previous edi- 
tions of the exams. This process allows the scaled scores reported to Naval Postgraduate 
School to be compared over time with scaled scores of other students taking older ex- 
ams. The only exception to this is that analytical scores after 1981 are not comparable 
to those before the 1981 revision. [Ref 16: p. 31] 

c. Percentiles 

The percentile ranks reported along with the scaled scores are not interpre- 
table over time. These rankings are made for a specific group of test takers. Educational 
Testing Service provides a table of percentile ranks which can be used to interpret 
rankings of test scores earned before 1986-87. [Ref 16: p. 31] 
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d. Guidelines 

Educational Testing Service provides guidelines to administrators for the 
use of the Graduate Record Examination scores. They suggest that the Graduate Re- 
cord Examination has two limitations; 1) it cannot measure all qualities which make 
up an individual and contribute to academic success and 2) "only score differences of 
certain magnitudes are reliable indicators of real differences in performance." [Ref 16: 
p. 14] Throughout the Guide Educational Testing Service cautions against using the 
Graduate Record Examination Scores as the only criterion for admission or selection. 
There are many factors besides test scores which should be considered. They also cau- 
tion against using a minimum (cut-ofO score philosophy for the same reason. [Ref 16] 

e. The Graduate Record Examination as an Admissions Standard 

A number of papers have been written discussing the Graduate Record 
Examination used as an admissions standard. Oltman and Hartnett indicated that about 
64 percent of the graduate programs either require or recommend Graduate Record 
Examination scores. They also surveyed administrators to find out how the Graduate 
Record Examination scores were used. Most respondents who use the exam scores in 
their admission process do so to supplement other, perhaps less satisfactory academic 
credentials. Undergraduate grades were consistently rated as the most important ad- 
missions factor. [Ref 17) 

f. The Graduate Record Examination as a Grade Predictor 

There has also been some concern about the validity of the Graduate Re- 
cord Examination as a predictor of graduate grades. Wilson provides a good discussion 
of the subject based on the results of "The GRE Cooperative Validity Studies Project" 
conducted by Educational Testing Service in the latter half of the 1970s. The biggest 
concerns in past studies were: 1) most of the samples were quite small, making results 
unreliable and 2) various institutions had different ideas about what "success" in gradu- 
ate school actually was. Wilson reports, however, that throughout the Validity Studies, 
grade point average was the common factor and was usually positive. He also stated 
that the Graduate Record Examination was generally a better predictor of graduate 
grades than other factors studied. [Ref 18] 

C. METHODOLOGY 

Statistical procedures in this thesis will focus primarily on exploring correlations 
between the Graduate Record Examination and Academic Profile Code indices and the 
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Graduate Quality Point Rating. The goal is to determine which measures are most 
strongly and significantly correlated with Graduate Quality Point Ratings. 

Additionally, the various indices will be regressed against Graduate Quality Point 
Rating. The resulting equation can be used by graduate education selection boards as 
a predictor of academic success. 

The statistical package which will be used for this analysis is SPSS^ . SPSS^ is a 
trademark of SPSS, Incorporated. 
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IV. DATA 



The data analyzed were available in the Naval Postgraduate School Admissions and 
Registrar olTices. A data set was entered manually from Graduate Record Examination 
score cards received at the Naval Postgraduate School from Educational Testing Service. 
Data included on these cards were social security number, date of birth, date of exam 
and the three Graduate Record Examination scores with their respective percentiles. 
This file was merged with four separate registrar files containing Academic Profile Code, 
curriculum, designator, rank, undergraduate information and grades earned at the Naval 
Postgraduate School. To protect the privacy of individual students, names were not in- 
cluded in any of the data. 

These data included only U.S. Na \7 students. The study has excluded other L'.S. 
service students as well as international students. 

To make the data more useful, several computations were performed. Age at time 
of exantination was computed by subtracting date of birth from the date of the exam. 
Time away from undergraduate studies was computed by subtracting date of under- 
graduate degree from date of exam. 

Since grading varies somewhat from department to department, cumulative Gradu- 
ate Quality Point Ratings were standardized to make them a better comparison measure. 
Standardizing allows for variations in department grading practices. The standardized 
scores were computed by dividing students' Graduate Quality Point Rating by a de- 
partment factor. The individual department factors were derived by dividing the de- 
partment averages by the overall school average Quality Point Rating of 3.45 (which has 
remained relatively stable over several academic years) [Ref 19]. Although the averages 
used were derived from all courses, not just 3000- and 4000-level courses, the numerical 
paucity of lower-division courses presents little effect on the factors computed. The re- 
sulting department factors are presented in Table 5 on page 15. 

The merged file contained 786 cases. Six of the original Graduate Record Exam- 
ination cases could not be matched with registrar files and were removed. The remaining 
780 cases provide some interesting descriptive information about the U.S. Na\T student 
population over the first 18 months of the study. 
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Table 5. DEPARTMENT STANDARDIZING FACTORS 



Code 


Department 


Factor 


52 


Computer Science 


1.029 


53 


Mathematics 


1.009 


54 


Administrative Sciences 


.980 


55 


Operations Research 


.994 


56 


National Security Affairs (relatively easy grading) 


1.067 


61 


Physics (relatively tough grading) 


.951 


62 


Electrical & Computer Engineering 


.986 


63 


Meteorology 


1.000 


67 


Aeronautics 


1.017 


68 


Oceanography 


1.006 


69 


Mechanical Engineering 


1.006 


71 


Antisubmarine Warfare Academic Group 


1.000 


72 


Electronic Warfare Academic Group 


1.044 


73 


Space Systems Academic Group 


1.044 


74 


Command, Control & Communications Academic Group 


1.052 




NPS--A11 Departments 


1.000 



Since the study began in April 1986, there have been seven administrations of the 
Graduate Record Examination for which data were available. These are shown, along 
with number of participating students in Table 6. 



Table 6. EXAM ADMINISTRATIONS 



Date of Exam 


Number of Examinees 


April 1986 


66 


August 1986 


148 


October 1986 


133 


Februarv 1987 


103 


April 1987 


97 


July 1987 


140 


October 1987 


93 


Total 


780 
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Students in this study are overwhelmingly male; 84.4 percent. Most students are 
either lieutenants or lieutenant commanders, 72.6 percent and 24.6 percent, respectively. 
They are 31 years old on average and have been away from undergraduate schooling an 
average of 7.6 years. 

By militar}' specialty, 66.7 percent are line officers (the majority being surface offi- 
cers (29 percent) and the minorities being special warfare, special operations and pilots 
(4.5 percent collectively)). Twenty percent are restricted line and the remaining 12.9 
percent are staff corps (mostly supply). 

The most popular curricula (the top 60 percent, listed in decreasing order) are com- 
puter systems, naval engineering, financial management, operations analysis, weapons 
systems engineering, electronic systems, air-ocean sciences, computer science, space 
systems operations and antisubmarine warfare. 

Descriptive statistics (shown in Table 7) regarding the Graduate Record Examina- 
tion, .'\cademic Profile Code and Standardized Quality Point Rating (ZQPR) scores re- 
veal the following information: 



Table 7. DESCRIPTIVE STATISTICS 



Variable 


.Vlean 


.Minimum 


.Maximum 


Standard 

Deviation 


Maximum 

Possible 

Range 


L QPR Code 


1.95 


0 


4 


.88 


0-5 


Math Code 


2.22 


0 


6 


1.13 


0-6 


Technical Code 


3.19 


0 


5 


1.57 


0-5 


G RE- Verbal 


547.95 


300 


800 


92.41 


200-800 


GRE-Quantitative 


632.03 


370 


800 


93.22 


200-800 


GRE-Analytical 


584.92 


260 


800 


103.31 


200-800 


ZQPR 


3.44 


0.00 


4.21 


.38 


0-4.27 



Although the preceding information is ver}' interesting and useful, there is concern 
about using this data for rigorous analysis. .Most students have very few, if any, 
graduate-level courses in their first quarter or two. In most curricula the courses build 
in difficulty as the program progresses. Additionally, as a student progresses through 
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the curriculum, its many courses and varied professors, the Quality Point Rating be- 
comes a more accurate measure of overall academic ability. It is therefore unfair to 
compare the Graduate Quality Point Rating of a first-quarter student with that of a 
si.xth-quarter student. By selecting those students who have completed exactly six 
quarters, one captures those who have graduated from 18-month curricula as well as 
those who have completed the majority of long curricula. These cases should provide a 
better measure of academic achievement. 

There is some concern among academics that there has been a steady grade creep 
over the years. In other words, a 25-year-old graduate student may have a higher 
undergraduate grade point average than a 40-year-old graduate student, yet be academ- 
ically equivalent. To attempt some limited correction for this, one case in which age at 
time of examination was 40 or greater was eliminated. 

By selecting only those records in which students had completed 6 quarters and were 
less than 40 years old, the data set was reduced to 198 cases. These 198 cases are de- 
scribed in the following tables. 

Table S shows the breakdown by sex. Again, the vast majority of the students are 
men. 



Table 8. SEX-SIXTH-QUARTER DATA 



Sex 


Percent 


.Male 


80.8 


Female 


19.2 



Military Specialties have been grouped into the three major categories of line, re- 
stricted line and staff corps as shown in Table 9 on page 18. 

Only three rank categories are represented in this data: lieutenant junior grade (one 
percent), lieutenant (71.2 percent) and lieutenant commander (27.8 percent). 

By limiting the data to those who have completed six quarters, the number of test 
administrations with usable data is limited to the five shown in Table 10 on page 18. 
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Table 9. MILITARY SPECIALTY--SIXTH-QUARTER DATA 



Specialtv 


Percent 


Total Percent 


General Unrestricted Line 


14.6 




Surface 


21.7 




Subsurface 


6.1 




Special Warfare, Special Operations 


1.5 




Aviation 


14.1 




Total Line 




58.0 


Total Restricted Line 




22.1 


Supply Corps 


14.1 




Other Staff Corps 


5.0 




Total Stair Corps 




19.1 


Total 




99.2 


Total column does not add to 100% due to missing 


cases. 





Table 10. EXAM ADMINISTRATIONS-SIXTH-QUARTER DATA 



Date of Exam 


Number of Examinees 


April 1986 


11 


August 1986 


60 


October 1986 


111 


Februarv 1987 


13 


April 1987 


3 


Total 


198 



Table 11 on page 19 lists the curricula, showing the percentage of those 198 stu- 
dents enrolled in each. 

Descriptive statistics were computed for this sample. Table 12 on page 20 includes 
the mean, minimum, maximum and standard deviation for the continuous variables. 
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Table 11. CURRICULUM ENROLLMENT-SIXTH-QUARTER DATA 



Curriculum 


Percent 


Computer Systems 


11.6 


Financial Management 


8.6 


Material Logistics Support Management 


7.6 


Naval Engineering 


6.6 


Telecommunications Systems Management 


6.6 


Electronic Warfare 


6.1 


Operations Analysis 


4.5 


Space Systems Operations 


4.5 


Antisubmarine Warfare 


4.0 


Manpower, Personnel & Training Analysis 


3.5 


NSA-N'uclear Strategic Planning 


3.5 


Weapons Systems Engineering 


3.5 


Acquisition & Contract Management 


3.0 


Intelligence 


3.0 


Computer Science 


2.5 


Air-Ocean Sciences 


2.0 


Communication Engineering 


2.0 


Operational Logistics 


2.0 


Transportation Logistics Management 


2.0 


.Aeronautical Engineering 


1.5 


NSA-lnternational Organizations & Negotiations 


1.5 


Space Systems Engineering 


1.5 


Transportation Management 


1.5 


Underwater Acoustics 


1.5 


Aeronautical Engineering 


1.0 


Electronic Warfare 


1.0 


Weapons Systems Science 


1.0 


Command. Control & Communications 


.5 


NSA-Far East, Southeast Asia, Pacific 


.5 


NSA-Europe. USSR 


.5 


Systems Inventor}' Management 


.5 
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Table 12. DESCRIPTIVE STATISTICS-SIXTH-QUARTER DATA 



Variable 


Mean 


Age 


31.05 


Time Since L'ndergrad Degree 


7.65 


UQPR Code 


1.89 


Math Code 


2.41 


Technical Code 


3.27 


GRE-Verbal 


545.46 


GRE-Quantitative 


627.42 


GRE-Analytical 


588.38 


Standardized Graduate QPR 


3.48 



Minimum 


Maximum 


Standard 

Deviation 


25 


39 


3.47 


1 


17 


2.95 


0 


4 


.832 


0 


6 


1.25 


0 


5 


1.66 


300 


740 


93.62 


370 


800 


95.39 


300 


800 


99.60 


2.52 


4.07 


.294 
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V. ANALYSIS 



To develop an understanding of the relationship between the standardized Graduate 
Quality Point Rating and the various explanatory' variables, it was useful to first look 
at simple correlations. The larger the correlation coefficient, the stronger the relation- 
ship. To further provide a better selection tool, regressions were run to estimate pre- 
dictive equations. 

To simplify discussion of the results, variables have been assigned the following 
names; 



Academic Profile Code 

Undergraduate Quality Point Rating Code APCl 

.Math Code APC2 

Technical Code APC3 

Graduate Record Examination 

Verbal VB 

Quantitative QT 

Analytical AN 

Other Variables 

Age at time of exam (in years) AGE 

Time since undergraduate degree (in years) T 

Standardized Graduate Quality Point Rating ZQPR 

Predicted Graduate Quality Point Rating QPR'" 



A, CORRELATION 

Using the SPSSx Pearson correlation function, correlation tables were constructed 
for the entire sixth-quarter data set as well as separately for the Policy and Information 
Sciences Division and the Sciences and Engineering Division. 

I. Sixth-Quarter Data 

Table 13 on page 22 depicts the relationship between ZQPR and three Gradu- 
ate Record Examination measures, the three Academic Profile Code measures, AGE and 
T. 

What becomes apparent in analyzing this table is that AGE and T are strongly 
correlated with one another and that T has a very' small effect on ZQPR. Not only is 
the correlation a small absolute value, but it is significant only to the ten percent level. 
The other anomalies in this table are APC2 and APC3. Both are quite small and sig- 
nificant only to the ten percent level for APC2 and the five percent level for APC3. The 
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Table 13. CORRELATION TABLE-SIXTH-QUARTER DATA 



AGE 


T 


VB 


QT 


AN 


APCl 


APC2 


APC3 


ZQPR 


AGE 1.000 
(198) 


.719 

(189) 

.000 


.023 

(198) 

..)70 


-.264 

(198) 

.000 


-.269 

(198) 

.000 


.042 

(194) 

.282 


.296 

(194) 

.000 


.230 

(194) 

.001 


-.244 

(198) 

.000 


T 


1.000 

(189) 


.150 

(189) 

.020 


-. 1 .)S 
(189) 
.029 


-.171 

(189) 

.009 


-.067 

(185) 

.183 


.233 

(185) 

.001 


.214 

(185) 

.002 


-.096 

(189) 

.094 


VB 




1.000 

(198) 


.272 

(198) 

.000 


.446 

(198) 

.000 


-.245 

(194) 

.000 


-.010 


-.042 

(194) 

.278 


.311 

(198) 

.000 


QT 






1 .000 
(198) 


.615 

(198) 

.000 


-.262 

(194) 

.000 


-.488 

(194) 

.000 


-.533 

(194) 

.000 


.424 

(198) 

.000 


AN 








1.000 

(198) 


-.124 

(194) 

.042 


-.233 

(194) 

.001 


-.269 

.000 


.350 

(198) 

.000 


APCl 










1.000 

(194) 


.164 

(194) 

.011 


.090 

(194) 

.107 


-.348 

(194) 

.000 


APC2 












1 .000 
(194) 


.562 

(194) 

.000 


-.108 

(194) 

.067 


APC3 














1.000 

(194) 


-.121 

(194) 

.046 


ZQPR 
















1.000 

(198) 


Correlation Coefficient 
(Number of Cases) 

One-Tailed Significance Level, p- value 











remaining variables: AGE, VB, QT, AX and APCl show reasonably strong correlations 
which are significant at the one percent level. 

The probable explanation is that the Graduate Record Examination quantita- 
tive and analytical sections are better measures of the math and science knowledge of 
an individual than the Academic Profile Code measures. 

The negative sign of the age factor suggests that "older" students do have 
slightly lower scores than their younger counterparts. 



These correlation results hint that T, APC2 and APC3 do not contribute much 
to prediction of graduate success. 

The reader must remember Table 2 on page 9, Table 3 on page 10 and 
Table 4 on page 10 when interpreting correlation coefficients of the Academic Profile 
Code measures. The lower the number in the code, the better. One would, therefore, 
expect negative correlations with ZQPR. The Graduate Record Examination scores, on 
the other hand, are "normal"--the higher the score, the better. These correlations are 
expected to be positive. 

2. Division Data 

Additionally, tables were constructed for the two major Divisions within the 
school. The correlation results are displayed in Table 14 on page 24 and Table 15 on 
page 25. 

a. Policy and Information Sciences Division (05) 

For the Policy and Information Sciences Division (X= 135) the table shows 
similar results. T is now a small and insignificant factor. QT and APCl appear to have 
the strongest correlation with ZQPR. APC2 and APC3 are small and not as statistically 
significant as the GRE measures. 

b. Sciences and Engineering Division (06) 

In the Sciences and Engineering Division (X = 63) T, APC2 and APC3 are 
relatively small and insignificant. AGE appears to be a much bigger factor for this 
group. Perhaps that can be explained by the theorv’ that quantitative skills decline as 
one gets older and does not use them as much. Of interest, though, is the very strong 
correlation between ZQPR and QT: .5940 which is statistically significant at the one 
percent level. As might be expected for the scientists and engineers, VB is not nearly as 
strong as QT and AX. 

c. Conclusion 

Table 16 on page 26 combines the previous three correlation tables into 
one. Shown are the correlations between ZQPR and the eight explanatory variables 
discussed. 

These results confirm that the Graduate Record Examination measures 
along with APCl and AGE are the most useful in predicting success at Xaval Post- 
graduate School. 
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Table 14. CORRELATION TABLE-DIVISION 05 



AGE 


T 


VB 


QT 


AN 


APCl 


APC2 


APC3 


ZQPR 


AGE 1.000 
(135) 


.709 

(129) 

.000 


-.018 

(135) 

.416 


-.285 

(135) 

.000 


-.271 

(135) 

.001 


.070 

(131) 

.215 


.343 

(131) 

.000 


.293 

(131) 

.000 


-.178 

(135) 

.019 


T 


1.000 

(129) 


.102 

(129) 

.126 


-.099 

(129) 

.la3 


-.126 

(129) 

.078 


-.017 

(125) 

.426 


.169 

(125) 

.030 


.210 

(125) 

.009 


-.049 

(129) 

.292 


VB 




1.000 

(135) 


.295 

(135) 

.000 


.491 

(135) 

.000 


-.306 

(131) 

.000 


-.078 

(131) 

.187 


-.164 

.031 


.341 

(135) 

.000 


QT 






1.000 

(135) 


.630 

(135) 

.000 


-.296 

(131) 

.000 


-.425 

(131) 

.000 


-.520 

(131) 

.000 


.438 

(135) 

.000 


AN 








1.000 

(135) 


-.152 

(131) 

.042 


-.281 

.001 


-.346 

(131) 

.000 


.335 

(135) 

.000 


APCl 










1.000 

(131) 


.144 

(131) 

.050 


.198 

( 131 ) 

.1 


-.431 

(Ul) 

.000 


APC2 












1.000 

(131) 


.523 

(131) 

.000 


-.160 

(131) 

.034 


APC3 














1.000 

(131) 


-.170 

(131) 

.026 


ZQPR 
















1.000 

(135) 


Correlation Coefficient 
(Number of Cases) 

One-Tailed Significance Level, p- 


•value 











B. REGRESSION 

SPSSx was used to perforin regression analysis on the entire sixth-quarter data set 
as well as the two major divisions. The intent was twofold. One objective was to de- 
termine how much of the variance of a predicted Graduate Quality Point Rating could 
be explained by the variables of interest. The second objective was to build an equation 
which could be used by selection boards to predict academic success at the Naval Post- 
graduate School. 
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Table 15. CORRELATION TABLE -DIVISION 06 



AGE 


T VB 


QT 


AN 


APCl 


APC2 


APC3 


ZQPR 


AGE 1.000 
(63) 


.732 .080 
(60) (63) 
.000 .268 


-.180 

(63) 

.079 


-.259 

!o2^ 


-.020 


.172 

(63) 

.088 


.066 

(63) 

.305 


-.348 

(63) 

.003 


T 


1.000 .226 

(60) (60) 

.041 


-.137 

(60) 

:i48 


-.238 


-.177 


.353 

(60) 

.003 


.120 

(60) 

.181 


-.170 

(60) 

.097 


VB 


1.000 

(63) 


.463 

(63) 

.000 


.390 

(63) 

.001 


-.103 

(63) 

.210 


-.032 

(63) 

.403 


.028 

(63) 

.414 


.286 

(63) 

.011 


QT 




1.000 

(63) 


.699 

(63) 

.ooO 


-.236 

(63) 

.031 


-.294 

(63) 

.010 


-.231 

(63) 

.034 


.594 

(63) 

.000 


AN 






1.000 

(63) 


-.070 

(63) 

.294 


-.154 

(63) 

.114 


-.159 

(63) 

.107 


.384 

(63) 

.001 


APCl 








1.000 

(63) 


.362 

(63) 

.002 


.113 

(63) 

.189 


-.263 

(63) 

.019 


APC2 










1.000 

(63) 


.198 

(63) 

.060 


-.160 

(63) 

.105 


APC3 












1.000 

(63) 


-.167 

(63) 

.095 


ZQPR 














1.000 

(63) 


Correlation Coefficient 
(Number of Cases) 

One-Tailed Significance Level, p-value 











I. SLxth-Quarter Data 

Using the "enter" method, a regression of the Academic Profile Code was run 
against ZQPR. The intent was to see how well the Academic Profile Code predicted 
QPR*. Second, a regression equation was formed with just the three Graduate Record 
Examination scores. In order to see whether combining the two sets of measures would 
be a better prediction tool, a third equation was developed with all the Academic Profile 
Code and Graduate Record Examination measures as explanator}’ variables and ZQPR 
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Table 16. COMBINED CORRELATION TABLE 



CoefTicients = correlations between ZQPR & the eight variables shown in this table. 
'•■■One-tailed significance level = .000 



GRE APC Other 





VB 


QI 


AX 


i 


2 


3 


AGE 


I 


Sixth-QTR 


.31P 


.424* 


.350"' 


-.348* 


-.108 


-.121 


-.244* 


-.096 


Div 05 


.341’^ 


.438* 


.335* 


-.431* 


-.160 


-.170 


-.178 


-.049 


Div 06 


.286 


.594* 


.384 


-.263 


-.160 


-.167 


-.348 


-.170 



as the dependent variable. Finally, based on the earlier deductions, an equation was 
formed using VB, QT, AX, APCl and AGE to explain ZQPR and, hence, QPR"^'. 

The hypothesis for this approach was that R* and R- would both increase with 
successive equations, while the confidence inten’al around the estimate would decrease. 
It is understood that R‘ will always increase with the addition of new variables. R^ , 
which is R- adjusted for degrees of freedom, will increase only if the new variable con- 
tributes to the equation. 

The results of these equations are summarized in Table 17. 



Table 17. REGRESSION RESULTS-SIXTH-QUARTER DATA 



Variables 


R^ 


R^ 




Acadeiriic Profile Code 
APCl, APC2, APC3 


.129 


.115 


.277 


Graduate Record Examination 
VB, QT. AX 


.223 


.211 


.261 


APC and GRE Combined 

APCl, APC2, APC3, VB, QT, AX 


.276 


.253 


.254 


Selected Variables 

APCl, VB, QT, .AX, AGE 


.289 


.270 


.252 



The Academic Profile Code alone explains only 13 percent of the variance in 
QPR•■'^ The Graduate Record Examination alone predicts 22 percent. When combined, 
R^ rises (as expected) to 27.6 percent. Interesting is that R^ increases as well. Finally, 
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the variables chosen as the best predictors increase to 28.9 percent and increase 
to 27 percent. 

Forecast intervals were developed using Equation (5.1) below. 



FI= Yj~^y±Spt^ 



(5.1) 



Where FI 

A 

>v., 

h 



the forecast interval 
the forecast 

the estimated standard error of the forecast 

the critical t value (in this case, 95 percent confidence) [Ref 20] 



For illustration, }'V-i = 3.479 (the mean ZQPR) for all equations, = 1.96 and 
Sf can be found in Table 17 on page 26 as SE. 

Computations reveal an ever-narrowing interval around the mean. See 
Figure 1 on page 28. These narrowing intervals combined with the good results of the 
regressions confirm the final set of variables in Table 17 on page 26 as the best predic- 
tors of success at Naval Postgraduate School. 

The prediction tool for selection boards to use would be the regression equation 
using VB, QT, AX, APCl and AGE shown in the order they entered the regression 
equation: 

= 3 . 2 1 85 - .0 1 3 7/1 G£(/« years) + .0006 VB 

- .0S15/1PC1 + .0007(77'-!- .OOOU.V 

The results will yield pPR^'-a forecast of the officer's Graduate QPR at the Naval 
Postgraduate School. QPR* will still be a standardized value. If a potential student's 
curriculum is known, QPR* can be multiplied by the appropriate department factor 
(Table 5 on page 15) to obtain a sharper estimate for that department. See Appendix 
A for which curricula belong in the departments. 

2. Illustration of Prediction Equation for Admission to NPS 

To illustrate the process, assume Lieutenant Junior Grade Grad is being con- 
sidered for graduate education. He is 27 years old. He graduated from American Uni- 
versity with a 3.00 grade point average. His Graduate Record Examination scores are 
550 verbal, 600 quantitative and 580 analytical. He is interested in studying in the 681, 
847 or 532 curricula. 
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< 


3.48 > 


2.94 


APC only 4.02 


2.97 


GRE onlv 3.99 


2.98 


— - APC and GRE 3.98 


2.99-- 


• — APC1,VB. 3.97 




QT, AN, AGE 



Figure 1. Forecast Intei^'als--SL\th-Quarter Data 



Using Equation (5.2), the selection board would discover a potential Graduate 
QPR at the Naval Postgraduate School of 3.494. See Equation (5.3). 

QPR* = 3.2185 - .0137(27) + .0006(550) 

- .0815(2) + .0007(600) + .0001(580) = 3.494 

This predicted Graduate QPR can then be adjusted for the specific curricula Lieutenant 
Junior Grade Grad is interested in: 
for curriculum 681: adjusted QPR* = 3.494(1.067) = 3.73 
for curriculum 847: adjusted QPR* = 3.494(.980) = 3.42 
for curriculum 531: adjusted QPR* = 3.494(.951) = 3.32 

The board could further analyze this officer's potential at the Naval Postgrad- 
uate School by putting a forecast interval around each estimate. This would allow the 
board 95 percent confidence that the officer's Graduate QPR would fall within the range 
indicated. 

For curriculum 681: FI = 3.73 ± 1.96(.252); that is, 3.24 ■< >• 4.22 

For curriculum 847: FI = 3.42 ± 1.96(.252); that is, 2.93 < >• 3.91 

For curriculum 531: FI = 3.32 ± 1.96(.252); that is, 2.83 ^ 3.81 

Lieutenant Junior Grade Grad may not be as good a risk in the physics curric- 

ulum as in national security affairs, or even in manpower, personnel and training 
analysis. 
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3. Division Data 



Using the same procedures used for the entire data set, regressions were com- 
puted for Divisions 05 and 06. The premise is that the two Divisions may have sub- 
stantially different types of students. 

a. Policy and Information Sciences Division (05) 

Table 18 shows regression results for Division 05. The results are quite 
similar, bearing out the choice of APCl, VB, QT, AN and AGE as predictors. 



Table 18. REGRESSION RESULTS-DIVISION 05 



Variables 




R^ 




APCl. APC2, APC3 


.203 


.185 


.217 


VB. QT, AX 


.241 


.224 


.212 


APCl. APC2, APC3, VB, QT, AN .309 


.276 


.204 


APCl, VB, QT, AN, AGE 


.312 


.285 


.203 


Slightly narrower forecast intervals arc 


evident in Figure 2. These intervals 


are also built around the mean ZQPR of 3.48. 






< 


3.48 




»> 


3.05 


-- APC only — - 


3.90 




3.06 


— GRE only — 


3.89 




3.07 


• APC and GRE 


3.88 




3.08 


- APCl.VB — - 
QT, AN, AGE 


3.87 





Figure 2. Forecast Intervals—Division 05 



Equation (5.4) is the prediction tool for this submodel. It could be used if 
the selection board were assured an officer would be interested in only Policy and In- 
formation Sciences curricula. 



QPR*- = 3.1520 - .0Q52AGE{in years) + .0004 UB 

- .08 14/1 PCI -b .0007^7 -b .0002,'! A 



(5.4) 
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b. Sciences and Engineering Division (06) 

Table 19 on page 30 contains regression results for Division 06. The results 
for this subgroup are considerably stronger: = .4440 as compared to .3123 for Divi- 

sion 05. 



Table 19. REGRESSION RESULTS-DIVISION 06 



Variables 


R^ 


R^ 




APCl. APC2, APC3 


.091 


.044 


.376 


VB, QT, AX 


.355 


.322 


.317 


APCl, APC2, APC3, VB, QT, AN 


.374 


.307 


.320 


APCl, VB, QT, AX. AGE 


.444 


.395 


.299 



Forecast intervals for Division 06 are considerably wider than either the 
overall data or Division 05 data. See Figure 3. The wider intervals are due to larger 
standard errors in this subgroup. One possible explanation is the smaller number of 
cases in this sample combined with the different type of students. These intervals, again 
are built around the mean ZQPR of 3.48. 



< 


3.48 


> 


2.74 


■ APC only 


4.22 


2.86 


GRE only 


4.10 


2.85 


— APC and GRE - 


4.11 


2.89- 


APCl, VB. 

QT, AX, AGE 


—4.07 



Figure 3. Forecast Intervals— Division 06 



Equation (5.5) is the prediction tool for this submodel. It could be used if 
the selection board were assured an officer would be interested in only Sciences and 
Engineering curricula. 

QPR* = 2.7122 - .0308/lG£(//j years) - .0693APCI 

+ .0004 VB - .0004^1 iV + .002807 
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c. Conclusion 



The Division regressions confirm the choice of variables for predicting 
Graduate QPR, namely, VB, QT, AN, APCl and AGE. in the Division computations 
is stronger in both cases than the overall regression. The results are strongest for Divi- 
sion 06. 

Using the individual Division prediction equations would be limiting to a 
selection board since the board would have to know exactly which curriculum an officer 
would be assigned to. A better tool would be Equation (5.2) which would allow a se- 
lection board to predict Graduate QPR for any officer. The department factor could 
always be applied for individual curricula-perhaps during the assignment process. 

QPR* = 3.2185 - .0137.4G£(/>» years) + .0006 VB 

- .0815.^RC1 -I- M\MQT+ .OOOITA^ 



Where QPR"- the predicted Graduate QPR 
AGE = the officer's age 

VB = the Graduate Record Examination Verbal measure 
APC1= the Undergraduate QPR 

QT = the Graduate Record Examination Quantitative measure 
AN = the Graduate Record Examination Analvtical measure 



C. DEPARTMENTS 



One of the original research questions was "Is there a difference in predictive value 
for different curricula?" In narrowing the specification of the data set, N became very 
small for individual curricula: range = 0 to 23. These small sample sizes would render 
computed statistics unreliable. 

Correlation and regression computations were made for the two departments in 
which N was greater than or equal to 25. Results are outlined in Appendix B. 



D. HOW DOES NAVAU POSTGRADUATE SCHOOL COMPARE? 

Educational Testing Service provides a means whereby an institution's students' 
scores on the Graduate Record Examination may be compared (in percentile fashion) 
to earlier test-takers. Table 20 on page 33 displays percentile ranks and scaled scores 
for General Test examinees between 1981 and 1985. Added in the right four sets of 
columns are mean scores for all Naval Postgraduate School students who have partic- 
ipated in the study so far, sixth-quarter students and the 05 and 06 Division students. 
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This table shows Naval Postgraduate School students well above the Educational 1 est- 
ing Service mean--particularly in the quantitative measure. 
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Table 20. GENERAL TEST INTERPRETIVE DATA 



Based on performa 


nee of General Test examinees between 1 Oct 81 & 30 Sep 84 




Percent of Examinees Scoring Lower Than Selected Scaled Scores 












All XPS 


6th-Otr 


Div 05 




Div 06 


Scaled 
















Scores VB 


Q1 


AX 


VB QT AX 


VB QT AX 


VB QT AX 


VB 


QT ^ 


SOO 




99 


99 














780 


99 


98 


98 














760 


99 


95 


97 














740 


98 


92 


96 














720 


96 


89 


94 














700 


95 


85 


92 














680 


93 


81 


90 












677 


660 


91 


77 


86 














640 


88 


73 


83 


632 




627 


j 






620 


85 


67 


79 




1 


1 


604 






600 


81 


64 


76 




585 


588 


585 




593 


580 


78 


58 


70 














560 


73 


53 


65 

1 


548 




545 


553 






540 


68 


48 


60 1 




j 






528 




520 


63 


43 


54 : 








1 


1 




500 


57 


38 


48 , 






' 


1 


1 




480 


52 


32 


37 1 








1 


1 




460 


46 


28 


37 1 














440 


40 


24 


30 i 














420 


35 


20 


26 














400 


29 


17 


21 














380 


25 


13 


17 














360 


20 


11 


13 














340 


16 


8 


9 














320 


12 


6 


7 














300 


10 


5 


4 














280 


7 


3 


3 














260 


4 


2 


2 














240 


3 


1 


1 














220 


1 


1 
















Mean 


472 


539 


505 
















N 


= 785,276 


N = 


780 


X = 198 • 


X = 135 




X = 63 


Source; 


Educational Testing Service Guide [Ref. 16; p. 17] & research data set means 
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VI. LIMITATIONS 



The results shown from this study may be influenced to an undetermined degree by 
a number of factors. 

Most of the students taking the test during this three-year study have been away 
from school for a number of years. Does that affect their Graduate Record Examination 
scores? The exam is administered after arrival at the school. Does that affect students' 
motivation, and subsequently, their performance? If motivation is high, would they 
study before taking the test? Would studying improve the scores? 

A. ADMINISTRATION OF THE GRADUATE RECORD EXAMINATION 

One of the concerns of the conduct of this study is the timing of the administration 
of the Graduate Record Examination. Most exairunees take the test in their senior year 
of college or shortly thereafter. Their purpose in taking the exam is to submit scores to 
graduate school as part of the admissions procedure. Their motivation is fairly well- 
defined: "The better I do on this test, the better my chances of getting into the school 
of my choice." 

During this study, the Graduate Record Examination is administered to incoming 
classes of students in their first few weeks of school. This is a time when students arc 
in the process of readjusting to academic life and can be very close to the time of the first 
midterm exams. Other than a personal pride and motivation to always do the best you 
can, there is no true incentive for doing well on the Graduate Record E.xamination. 
These test-takers have been accepted and are enrolled; the exam results have no bearing 
on their academic or professional future. 

There are two conflicting views on this potential problem. In his response to the 
Graduate Education Review Board tasking, the Superintendent of the Naval Postgrad- 
uate School indicated that the "GRE was given to students at the Postgraduate School 
about twenty years ago, but only after they arrived and consequently served no use in 
the selection process." [Ref 16] Barr and Howard acknowledged that the data being an- 
alyzed are of students who have already been selected based on Academic Profile Code 
and undergraduate and professional performance. They believe, however, "that this does 
not pose a serious problem." [Ref 12] 

Not mentioned in either case is the lack of some sort of control group. As the 
examinees in this study arc all a specially selected group, there is no way to compare 
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them to the "universe" containing all students. The selection process does not know 
whether Graduate Record Examination scores of those selected for the school are sig- 
nificantly higher than the scores of those not selected for graduate education. 

The author conducted an informal ad hoc survey of a number of classmates to de- 
termine whether they knew why they were taking the exam and what sort of effort they 
made in taking the test. Most of the classmates admitted a vague understanding of why 
they were taking the Graduate Record Examination. Many, however, indicated that 
they had not exerted as much efibrt as they would have had the exam been a require- 
ment for admission. None of the students had made an effort to study or otherwise 
prepare for the exam. 

B. OLDER STUDENTS 

Students selected for the Naval Postgraduate School are generally lieutenants or 
lieutenant commanders. This means they have been in the Navw at least four years and 
are generally in their mid- to late twenties. As revealed earlier, the test-takers in the 
current study average 31 years of age and had earned their undergraduate degree an av- 
erage of 7.6 years ago. 

Several researchers have studied the effect on Graduate Record Examination scores 
of being an "older student." Hartle, Baratz and Clark looked at test-takers who were 30 
years of age or older or who had graduated with an undergraduate degree eight or more 
years ago. They found that the sample had a greater number of women and minorities 
than expected. Their hypothesis was that these categories of people, for a number of 
reasons, do not go directly to graduate school from their undergraduate programs. They 
also found that choice of studies favored education and the humanities and social sci- 
ences. Their bottom line was that, on average, older students' Graduate Record Exam- 
ination scores were lower than younger students' scores-particularly in the quantitative 
portion of the exam. (Ref. 21] 

Clark discussed the subject further in 1984 and found that average verbal scores 
were about the same for both types of students. Quantitative scores, however, were 
generally lower. She further reported on a study conducted by Hartnett and Oltman in 
1983. They reported that overall, the verbal scores tend to be higher for "older" women 
and about the same for "older" men. The quantitative scores are usually lower for both 
men and women. [Ref 22] 

It remains for a further study to determine whether the scores--especially quantita- 
tive scores--should perhaps be weighted in some fashion to account for the "older" 
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students or whether the Graduate Record Examination should simply be administered 
at an earlier time in an ofilcer's career. 

C. PREPARATORY STUDY 

Swinton and Powers have studied the effects of coaching and preparatory’ study on 
the Graduate Record Examination for about six years. Their latest effort in 1985 con- 
cluded that any coaching or preparatory study beyond average student preparation for 
the Graduate Record Exantination would "probably not result in higher test scores." 
[Ref. 23: p. 23] This conclusion applied to all three measures of the Graduate Record 
Examination. They assume some test preparation (i.e., reading the GRE Information 
Bulletin) by the average student. Not only have Naval Postgraduate School test-takers 
not received the Bulletin prior to the exam, they have no real incentive to undertake even 
limited preparation. 
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VII. CONCLUSION 



A. SUMMARY 

The data reveal that the Graduate Record Examination is a much stronger predictor 
of graduate success than the Academic Profile Code. Even stronger is the Graduate 
Record Examination plus APCl and AGE. Although is not exceedingly large, this 
can be explained by the variety of other variables which can also be considered when 
making graduate selections. These are variables which would be veiy difficult to quan- 
tify: professional performance, promotability, militaiy’ career path, variances among 

undergraduate institution grading policies, personality factors, family status and IQ are 
but a few. 

B. RECOMMENDATIONS 

Based on the analysis conducted in this thesis, Navw should consider using the 
Graduate Record Examination in conjunction with undergraduate grade point averages 
and the age of the officer. These measures should supplement professional and military 
considerations in the selection process. Rather than trv'ing to establish a cutofT score for 
the Graduate Record Examination elements, selection boards should use Equation (5.2) 
to determine potential academic success at the Naval Postgraduate School. 

OPR- = 3.2 1 S5 - .0 1 37/^ GE{m years) -1- .0006 VD 

- M15APCI -f .0007Qr-f .OOOMA 



Where QPR"= the predicted Graduate QPR 
AGE = the officer's age 

VB = the Graduate Record Examination Verbal measure 
APC1= the Undergraduate QPR 

QT = the Graduate Record Examination Quantitative measure 
AX = the Graduate Record Examination Analytical measure 
This equation is recommended to simplify computations for the selection boards. 
The overall equation will allow boards to compute one equation for all candidates, re- 
gardless of choice of curriculum. The department factor should be applied later, during 
the assignment process, to better place students in appropriate curricula. 
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C. FURTHER RESEARCH 

Further research on this data should be conducted when the study is ofilcially com- 
pleted. This will increase the number of cases for observation and undoubtedly enrich 
the results. 

A cost-benefit analysis should be conducted to determine which of the measures is 
more economical. This should include the current Academic Profile Code procedures, 
administration of the Graduate Record Examination (or reimbursement for e.xams ad- 
ministered at non-Xa\T facilities) and record maintenance costs. 

The effect of the time and place of administering the exam is an important issue 
which must be addressed. The e.xam should be administered early in an officer's career 
in order to be available and current during the graduate education selection process. 

Further research needs to be made within the 15 departments. There is some evi- 
dence, even with this small data set, that predictions will vary across curricula. W'hen a 
more reliable set of equations can be formulated, they should be used to select students 
and to place them into appropriate curricula. 

If the Graduate Record Examination is administered close to the officer's commis- 
sioning. there is still a potential problem of "older" students. Officer Candidate School 
and Officer Indoctrination School students can be older at comniissioning than Reserve 
Officer Training Corps (ROTC) or Academy graduates. To make scores more equiv- 
alent. a weighting factor could be developed for the older examinees. 

In any case, the Graduate Record E.xainination should be a much better predictor than 
the Academic Profile Code alone. 



38 



APPENDIX A. NPS CURRICULA 



05 Policy and Information Sciences Division 

52 Computer Science Department 

367 Computer Systems Management 

368 Computer Science 

53 Mathematics Department 

380 Advanced Science 

54 Administrative Sciences Department 

813 Transportation Logistics Management 

814 Transportation Management 

815 Acquisition and Contract Management 
819 Systems Inventory Management 

827 Material Logistics Support Management 
837 Financial Management 
847 Manpower, Personnel and Training Analysis 
620 Telecommunications Systems Management 

55 Operations Research Department 

360 Operations Analysis 

361 Operational Logistics 

56 National Security Affairs Department 

681 Middle East, Africa, South Asia 

682 Far East, Southeast Asia, Pacific 

683 Europe, USSR 

684 International Organizations and Negotiations 

685 Western Hemisphere 

686 Strategic Planning - General 

687 Strategic Planning - Nuclear 
825 Intelligence 

06 Sciences and Engineering Division 
61 Physics Department 

530 Weapons Systems Engineering 

531 W'eapons Systems Science (Physics) 
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532 Nuclear Physics (Weapons and EfTects) 

535 Underwater Acoustics 

62 Electrical and Computer Engineering Department 

590 Electronic Systems Engineering 
600 Communications Engineering 

63 Meteorology Department 

372 Meteorology 

373 Air-Ocean Science 
67 Aeronautics Department 

610 Aeronautical Engineering 

61 1 Aeronautical Engineering - Avionics 
6S Oceanography Department 

374 Operational Oceanography 

440 Oceanography 

441 Hydrographic Sciences 

69 Mechanical Engineering Department 
570 Naval (Mechanical) Engineering 

71 Antisubmarine Warfare Academic Group 

525 Antisubmarine Warfare Systems 

72 Electronic Warfare Academic Group 

595 Electronic Warfare Systems Engineering 

73 Space Systems Academic Group 

366 Space Systems Operations 

59 1 Space Systems Engineering 

74 Command, Control and Communications Academic Group 

365 Joint Command, Control and Communications 
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APPENDIX B. DEPARTMENT STATISTICS 



52 Computer Science (N=28) 



Table 21. DESCRIPTIVE STATISTICS -DEPARTMENT 52 



Variable 


Mean 


.Minimum 


.Maximum 


Standard 

Deviation 


AGE 


31.43 


26 


39 


3.65 


T 


8.11 


4 


15 


2.99 


APCl 


2.12 


0 


4 


.950 


APC2 


2.54 


0 


5 


.990 


APC3 


3.65 


0 


5 


1.88 


VB 


536.43 


430 


670 


75.44 


QT 


596.07 


440 


800 


90.81 


AN 


578.93 


440 


770 


78.47 


ZQPR 


3.43 


2.86 


3.89 


.253 



Table 22. REGRESSION RESULTS-DEPARTMENT 52 

Variables SE 

APCl, VB, QT, AN, AGE .361 .201 .225 

Forecast Interv^al: 3.479 ± 1.96(.2254); that is, 3.04 •* — »• 3.92 

QPR* = 3.2185 - .OUl AG E{m years) + .0006V B 

- .08 15^1 PCI -I- .0007^7-1- .OOOl^iV 
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Table 23. CORRELATION TABLE-DEPARTMENT 52 



AGE 


T VB 


QT 


AN 


APCl 


APC2 


APC3 


ZQPR 


AGE 1.000 
(2S) 


.78S .400 

(27) (2S) 

.000 .017 


-.236 

(28) 

.113 


-.167 

(28) 

.198 


.125 

(26) 

.271 


.323 

(26) 

.054 


.343 

(26) 

.043 


-.260 

(28) 

.091 


T 


1.000 .341 

(27) (27) 

.055 


-.106 

(27) 

.299 


-.195 

(27) 

.164 


.073 

(25) 

.364 


.339 

(25) 

.049 


.308 

(25) 

.067 


-.061 

(27) 

.382 


VB 


1.000 

(28) 


.295 

(28) 

.064 


.303 

(28) 

.058 


-.160 

(26) 

.217 


-.019 

(26) 

.463 


.051 

(26) 

.403 


.088 

(28) 

.328 


QT 




1.000 

(28) 


.697 

(28) 

.000 


-.417 

(26) 

.017 


-.482 

(26) 

.006 


-.689 

(26) 

.000 


.481 

(28) 

.005 


AN 






1.000 

(28) 


-.132 

(26) 

.261 


-.255 

(26) 

.104 


-.279 

(26) 

.084 


.134 

(28) 

.249 


APCl 








1.000 

(26) 


.611 

(26) 

.000 


.472 

(26) 

.008 


-.453 

(26) 

.010 


APC2 










1.000 

(26) 


.601 

(26) 

.001 


-.562 

(26) 

.001 


APC3 












1.000 

(26) 


-.403 

(26) 

.021 


ZQPR 














1.000 

(28) 


Correlation CoefTident 
(Number of Cases) 

One-Tailed Significance Level, p- 


•value 
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53 Mathematics (N = 0) 
No statistics computed. 



54 Administrative Sciences (N=66) 



Table 24. DESCRIPTIVE STATISTICS-DEPARTMENT 54 



Variable 


.Mean 


.Minimum 


.Maximum 


Standard 

Deviation 


AGE 


31.58 


25 


37 


3.16 


T 


8.06 


3 


15 


3.00 


APCl 


1.83 


0 


4 


.883 


APC2 


3.00 


0 


6 


1.13 


APC3 


4.27 • 


0 


5 


1.19 


VB 


550.15 


300 


710 


92.89 


QT 


587.58 


370 


750 


89.31 


AN 


575.91 


340 


800 


96.36 


ZQPR 


3.53 


2.96 


3.97 


.239 


Table 25. REGRESSION RESULTS- 


-DEPARTMENT 54 






Variables 




R^ 


SE 


APCl, VB, 


QT, AN, AGE 


.353 


.297 


.201 


Forecast Interval: 3.479 ± 1.96(.2007); that is, 3.09 


3.87 





OPR- = 2.8066 + .0010/lGE(/«'v>.’cu«) - .0070TPC1 

+ .0002VB + .0007QT+ .0004.4 V 
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Table 26. CORRELATION TABLE-DEPARTMENT 54 



AGE T VB 


QT 


AN 


APCl 


APC2 


APC3 


ZQPR 


AGE 1. 000 .703 -.116 

(66) (65) (66) 

.000 .177 


-.261 

(66) 

.017 


-.291 

(66) 

.009 


.010 

(64) 

.467 


.336 

(64) 

.003 


.215 

(64) 

.044 


-.128 

(66) 

.153 


T 1.000 .110 

(65) (65) 

.192 


-.067 

(65) 

.298 


-.099 

(65) 

.217 


-.087 

(63) 

.248 


.198 

(63) 

.060 


.057 

(63) 

.328 


-.017 

(65) 

.446 


VB 1.000 

(66) 


.447 

(66) 

.000 


.536 

(66) 

.000 


-.226 

(64) 

.037 


-.082 

(64) 

.259 


-.244 

(64) 

.026 


.400 

(66) 

.000 


QT 


1.000 

(66) 


.662 

(66) 

.000 


-.214 

(64) 

.044 


-.246 

(64) 

.025 


-.334 

(64) 

.003 


.499 

(66) 

.000 


AN 




1.000 

(66) 


-.039 

(64) 

.380 


-.216 

(64) 

.044 


-.278 

(64) 

.013 


.439 

(66) 

.000 


A PCI 






1.000 

(64) 


-.016 

(64) 

.450 


-.168 

(64) 

.092 


-.374 

(64) 

.001 


APC2 








1.000 

(64) 


.392 

(64) 

.001 


-.141 

(64) 

.134 


APC3 










1.000 

(64) 


-.049 

(64) 

.350 


ZQPR 












1.000 

(66) 


Correlation CoefTident 
(Number of Cases) 

One-Tailed Significance Level, p-value 
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55 Operations Research (N=13) 

Xo statistics computed. 

56 National Security Affairs (X=18) 

No statistics computed. 

61 Physics (N=12) 

No statistics computed. 

62 Electrical and Computer Engineering (N=16) 

No statistics computed. 

63 Meteorology (N = 4) 

No statistics computed. 

67 Aeronautics (N=5) 

No statistics computed. 

68 Oceanography 

No statistics computed. 

69 Mechanical Engineering (N= 13) 

No statistics computed. 

71 Antisubmarine Warfare (N=8) 

No statistics computed. 

72 Electronic Warfare (N=2) 

No statistics computed. 

73 Space Systems (N=12) 

No statistics computed. 

74 Command, Control and Communications (N=l) 
No statistics computed 
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